Every traffic sign is designed in a way where it is visible irrespective of the background or time of the day. Different traffic signs vary based on thier shapes and colours with a shape or number present on the sign. Traffic sign recognition is an important part of driver assitance and to make self-driving vehicles. Classification of traffic signs is an important part of Advanced Driver Assistance Systems (ADAS) which allows the vehicles to Self-drive with particular set of rules and within a spped limit.
Traffic Signs are crucial for following proper rules on roads to ensure safety on the road. Self-Driving cars is an upcoming technology in the car industry which requires accurate functioning of Traffic Sign Recognition (TSR) System. Currently for the Self-Driving technology one of the core technologies is Traffic Sign Recognition(TSR) alongside Advanced Driving Assistant System(ADAS). TSR is also used in all cars launched after 2015 to alert the driver of the speed limit on the road. Road signs are recognised by the camera mounted in front of the car. In Germany, a TSR benchmarking competition is held to support the industrial development.
Traffic Signs are road signs placed on the sides of roads to instruct or inform the road users of the road conditions or the rules implemented on the road.
There are different types of traffic signs like:
Warning Signs
Prohibition Signs
Mandatory Signs
Warning Signs: To warn the road users of any hazardous conditions on the road
Prohibition Signs: Actions that are not permitted for that road
Mandatory Signs:
1) Detection: Identifying a traffic sign and ignoring the background at higher speeds
2) Classification: Differentiating the different traffic signs based on the colour and shape of the sign and then identifying the sign based on what is displayed
Different types of traffic signs are defined in the "label_names.csv" file
Continued -
Different types of preprocessing are performed on the data and saved as various pickle files.
The initial train dataset before preprocessing is done. This data is a dictionary with four keys.
The four keys of the train data are defined as -
'features' - Is a 4D array with raw pixel data of the traffic sign images,
(number of examples, width, height, channels).
'labels' - Is a 1D array containing the label id of the traffic sign image,
file label_names.csv contains id -> name mappings.
'sizes' - Is a 2D array containing arrays (width, height),
representing the original width and height of the image.
'coords' - Is a 2D array containing arrays (x1, y1, x2, y2),
representing coordinates of a bounding frame around the image.
Method 'astype' is used to convert ndarray from int to float.
The size of the train data for each keys is displayed -
x = d['features'].astype(np.float32) # 4D numpy.ndarray type, for train = (34799, 32, 32, 3)
y = d['labels'] # 1D numpy.ndarray type, for train = (34799,)
s = d['sizes'] # 2D numpy.ndarray type, for train = (34799, 2)
c = d['coords'] # 2D numpy.ndarray type, for train = (34799, 4)
The dataset used for the recognition and classification of data for this project is data2.pickle.
The data2.pickle file contains all the all the train, test and validation data used for the deep learning model.
Shape of train, test and validation data -
Some examples of training data in grid form -
Flow Chart of CNN -
Different Types of activation functions mainly used are -
Different types of Optimizers -
Effect of Various Learning Rate -
During the training of the data only using CNN network model, validation accuracy is close to zero.
This occurs due to overfitting of data.
Plot of the Initial Trained Model -
Training Accuracy of the Data is how accurately the data has been trained.
Validation accuracy of the data is the output accuracy. Output accuracy refers to how weel the model has recognised and classified the traffic sign images.
Values of all the Training and Validation Accuracy from the above Results -
Testing Accuracy is used to check how well the model has performed when compared to the validation accuracy.
Output of Some Sampled Data with Filter 3x3 -
With Filter 5x5 -
With Filter 9x9 -
With filter 13x13 -
With filter 15x15 -
Model 2 Summary -
Training Accuracy of model 2 -
Validation Accuracy of Model 2 -
Result Values of Training and Validation Accuracy of Model 2 -
Testing Accuracy of Model 2 -
Classification Time of Model 2 -
Output of Model 2 Sampled Data with Filter 3x3 -
Model 2 with Filter 5x5 -
Model 2 with Filter 9x9 -
Detection Sample Output -
Problems Faced:
Dirt is the biggest issue faced as the car will not be able to detect the sign or even cannot classify the sign based on the symbol present on the sign. Unable to read traffic signs causes unsafe driving behaviours.
Under heavy weather conditions like weather or rain, detection of traffic signs is a huge issue.